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ABSTRACT 

Emerging criteria are reported for judging the 
trustworthiness of action research studies as compared to the 
criteria established for judging the trustworthiness of other forms 
of naturalistic inquiry set forth by X. s. Lincoln and E. Guba 
(1985). Differing conceptions of the nature of action research are 
delineated, and their accompanying in^jlications for validity in 
action losearch are noted. Action science forms the theoretical basis 
for establishing these criteria for determining trustworthiness in 
action research. Additional validity criteria and issues for action 
research are offered via discussions of two studies: (1) a 1952 study 
by K. Lewin concerning eating habits; and (2) a study by K. Watkins 
concerning superviisor empowerment in a large company. Saps i»tween 
the theory and practice of validating action research projects point 
to the need to clarify and emphasize the significant role of 
teaching/learning and interpersonal competence in produci-'g valid, 
potentially generalizable, results in action research. Three figures 
and one table illustrate the discussi-on. A 3:»-itew list of references 
is included. (Author/SLD) 
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This paper reports emer^ng criteria for judgii^ the trustworthiness of action research 
studies as compared to the aiteria established forjudging the trustworthiness of other 
forms of naturalisdc inquiiy (Lincoln and Cuba, 1985). In this paper, differing 
c(HK;eptions of the nature of action research arc delin^ted and their accompanying 
implications for validity in action research iKHed Actiori science forms the theoretical basis 
for establishing criteria fQ€ detemnining tmstwovthiness in action research offered here. 
Criteria forjudging action science research in particular are o£fered with illustrations from 
two studies. 

Issues ir Action Research 
Action research is a form of research fint developed by Kurt Lewin who coined the t^m 
about 1944 (Keimnis, in Kemmis ami McTaggait» 19w). It has been criticized for either 
producing research with litde action ch* action with litUe research (Foster, 1972; Sanford, 
1981); weak when merely a form problem-sdving and strong when also emancipatory 
(Peten and Rd)inson, 1984; Kemmis, in Kemmis and M^Ti^gait, 1988), lacking the 
rigor of true scientific research (Coh^ and Manion, 1985), and lacking in internal and 
ext^tial control (Meniam and Simpson, 1984), hence of limited use in contributing to the 
body of knowledge. 

This paper is grounded in the rvork of Qiris Argyris who with Donald Schon 
developed a variation of action research. From Argyris* eariy work m the inner 
contradictions rigorous research (1980) to the later woik entitled Action Science 
(Argyris, Putnam, Snuth, 1985), Argyris has criticized other forms of research for 
producing knowledge which does not inform action. One of Kuit Lewin's last students, 
Argyris's vfoik extends Lewin's concepticm of action research to include a mediodology 
for transfimming individuals and organizations so tl^y can overcome the embedded patteir>s 
of reasoning which subvert action research/change programs. Argyris and Schon believe 
that social scientists are faced with a fundamental choice between rigw or relevance. 
Axgyxis (1980) criticized normal science for focusing on trivial problenas, for distorting 
both subjects and researehers, for generating unreliwle data, and for being generally unable 
to ans««r qucations about everyday life. 

On the other hand, Marris and Rein (in Cohen and Manion. 1985) aigue that die 
principles of action and research are so different as to be mutually exclusive so that to link 
them togetiier is to create a fundamental intmial conflict Sanfotd (198 1, in Reason and 
Rowan) argues Uiat the problem is one fiagmentation in social science which has led to 
a state of irrelevance such diat social scientists are among the last to be called upon to solve 
social and pdicy problems. Finally, Rappopon (1970) suggests that action research must 
resdve three dilemnnas, die resolution of dtter skle of which will exacerbate the otiter. die 
dilemmas of goals, ethics and initiatives. The goals of action and research may appear 
irreconcilable but must be balanced through such tilings as contracting for adequate time for 
the research, access to data, and ^propriate subjects. Etiiical issues include 
confidentiality, collusion witii the host c^anization, and competing organizations seeking 
access to the research or the researcho-. Dilemmas of initiative stem froin the fact that 
action research is service-oriented ami grows out of die initiatives of die p»^ulatio i under 
study radier than (solely) die initiative of die researcher which leads to problems of control. 
This is particularly significant when die host organization sees one type of problem and die 
research >'ields quite a different type of problem. Rneire (1982) drew a different line: "We 
have to be very clear about die objective of dus work: it is die people themselves, not the 
advancement of science. If, however, the people are silent, dicn we have to provoke tiiem, 
because we are not neutral" (in Kemmis and McTaggart, 1988, p. 272). These issues 
suggest that threats to validity differ in action research and diercfwe criteria for establishing 
validity must evolve out of die unique characteristics of action research. DitTerent 
conceptions of die practice of action research may help determine implicit validity criteria. 
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Defining action research, Kurt Lewin identified action reseaith as a cyclical 
pnxress which moves from an idea to reconnaissance to plan to action followed by 
reconnaissance* etc. (Lewin, 1952, in Swanson, Newcomb and Hartley). Otiieis have 
ucfined action research with a number oi diSoent onphases. 

"ActicHi research is research on action with the goal of making that action more 
effective" (French and Bell. 1988). Here ses^ research is measured by the difference in 
effectiveness of the subsequent actions takai. Reflection on action (SclKm, 1983, 1987) 
would by this definition be a form of actkm research. 

"Action research takes its cues-its questions, puzzles, and problems- fircsn the 
perceptions of practitioners witiun particular, local practk^e contexts. It bounds qn^es of 
research ^xording to die boundaries of the kx^ context It builds descriptions and 
tiiecaies within the practice ccm^t itself, and tests them throu^ intervention experiments- 
that is, through emoiments that bear the cknible burden of testing hypc^ses and effecting 
some feutatively) ctesiiablc change in the situatiai" (Argyris and Schan, 1991, p. 86). In 
this definition, the intm'cntions are the expmmental manipulation and problem-solving is 
the goal. Contribution to knowledge is in the area of research on intervention. 

"Action research is a fam erf coUeaivc self-reflective enquiry undertaken by 
I»rticipants in social situations in oder \o in^rcive the raticBiality and justice of their own 
sodal educational practu^s, as well as their understai^g of these practices and the 
situations in which tlwsc practices are carried out" (Kemmis and McTaggart, 1982, p, 5). 
Self-undostanding and transfonnation is the goal in this resean;h. Craitributions to 
knowledge are likely to be in die area of rdlective learning and strategic to improve the 
raticHiality and justice of educational practK^e. 

It's dear that die aliases in each of these (kfinitions arc quite different. Several 
streams of action research have emerged. Oiein, Cook and Harding (2948) suggest four 
types: diagnostic (in which ib& res^rcher/scientist diagnoses a problon ai^ makes 
recommoidations for change), participant (in which the acTfs learn to take action and to 
conduct research; this is now called paiticipatcHy action research, Whyte, 1991), empirical 
(in which the act<MS document action for research and reflecticHi) and experimental (in 
which there is contndled research on dififerent acticHi alternatives). Kemnus (in Kemn.is 
and McTaggart, 1988) finds that action research has become more a tedinology or a 
method than a means as Lewin first envisioned it to bring about social change. He 
advocates the use of action r^earch as a way of doing critical social sdence aimed at 
empowering people fhnii tradition, habit, bureaucratic systenuzati(», individual 
expectations, or frwn instrumentalisn. The fcdtowing three variants illustrate the way in 
which the differing definitional emphases lead to different ^)]»ioaches. 

D^erentiating between action research, action l&trning, and action science. All 
three are organizational dcvelqaneni aRjroach^ aimed at bringing about organizatio«al 
change. Differences between the tiirec versbns of action research might be summarized 
thus: 

•Action Research: Data infomis action 
•Action Laming: Learning improves action 
•Action Sdence: Meta l^uiniiig tran^orms action 

Action research grew out of the assumption tiiat developing data (the facts of die 
context) would lead to an unfieczing of the dynamic equilibrium jaescnt in people, groups 
and organizations. By collecting data around a jroblera and then feeding it back to die 
organization, die need for change would be crated while the duection that that change 
might take would also be scLf-cvident Reflection on problems in groups which typically 
include meiabas of the client organizatic^, the collection of data around their problems, 
group analysis and group feedback, groupndesigned interventions to attempt to solve die 
problem, an iterative process of int^vening, and die additional collection of data on die 
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effectiveness of the intervention fbUo^ved by reflection awi the inventbn of alternative 
intowiUi^S axe all hallmarks of the Lewin-invoited action researdi team. 

Action learmng adds the assun^tion that what ptevenis people from implen^ting 
thp changes that they sedc to make is die necessary skills, e^iecially the skill oi challenging 
pet assumptions and thus of critical reflection. Revans (1982) defines action learning as "a 
means o^velopnient, intellectual, ^notional, oi physical that requires its sub^t, 
thxmigh responsible involvement in some real, om^lex and stxessful problem, to achieve 
intended change sufficient to imjnove his ob^rvable behaviour henceforth in the problem 
fiekl" (pp. 6lS^). He triggers unfineezing of individuals by placing them in the situation 
that most m:tH>n researchers are in: they must solve real prob^ns in real contexts and Uie 
success or feilure of their efforts will be known and hi^ly visible (puUic). Group 
leftection is again a hallmark. Action learning teams <^n acki an element of sirangCT-i»s$ 
to hd^ten mtical reflectivity by having groups solve problems for dq>aiiments and 
organizations of whkh Uiey are not a part and of which they have tittle prior knowledge. 

Action science adds to each of die above tiieideatiiat wecan have a science of 
inteipersonal action dq}icted as Modell and Model n. In other w(»ds, there are tiungs we 
kiK>w about human interaction tiiat are repeatably true and Uius genoalisible, and some of 
these characteristics create barriers to tiie implementation of change. These characteristics 
aie learned, hi^y skillful respcmses to tiir^ and are intervenable by ^on scientists. 
Moreover, in learning to intervene in a way tiiat transforms peq>le and organizations so tiiat 
ongoing change is possible, action scientists are also adduig to the knowledge base and 
creating a new sdence cs dieoiy oi inteipa'sonal action. 

Each of these variations places an en[q}lmsis on one part of the action research 
process over another, yet there are some generally agreed upon characteristics. The 
characteristics of action research were examined m me works of eteven action researchers 
by Inters and R(^inson (1984). They identified twelve general characteristics and tiuee 
idiosyncratic characteristics, tiien assessed Uie presence a absence of each in tiie worics of 
the eleven action researchers. Those appearing most often were that the research was 
problem-focused (mentioned in 1 1 works), collaboratively conducted and participatory 
(1 1), action-oriented (1 1), an organic or cyclical process (8), and scientific (8). Oti^er 
cat^ories mentioned less often were terms that often overl^ped in meaning: nomiative, 
ethically-based, experimental, reeducative, emandpatoiy, stress on group dynamics, and 
naniralistic. This analysis, tiiough sometimes inaccurate, showed tiiat only Argyris and 
Lewin incoipraate all but one of tiiese characteristics. ^ Using tiieir classification, it is clear 
that flie research emphasis implied by "sdentific" or "experimental** has less priority even 
among many of these practitioners of action research. What, then, are the implications for 
validity of this type cf action research? 

Validity 

Validity itself has many forms. Brinbei^g and McGrath (1985) point out tiiat 
validity is not a commodity tiian can be obtained through technique... It is to be assessed 
relative to pmposes and drcumsta-'ces. They identify different stages of tiie research 
process and types of research and Swg^t tiiat validity criteria are (tifferent for each. 
Minimally, tiie concept of validity as value or worth is appropriate for defining the nature of 
the relationships to be studied; v^idity as correspondence or fit is appropriate as a test of 
the methodology and procedures for doing the snidy; while validity as robustness is a 



*Fi» example* Peters and Rc^inson imply ihai Argyris does noi stress a group dynamic, yet this is a 
cornerstone of ^tian science "a science of iruerpersoml action" which relies heavily on case groups to bring 
aboct change. 
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useful criteria for examining the intoimation of the data. Goetz and LeCompte (1984) 
discuss valkiiQr in qualitative designs and si^g^t that validity requires that we answer two 
primary quesdms. First» do the researchers {^mialiy observe and nKasuie whitt they say 
they are obsoving or measuring? Seccaid, to what extott are the findbigs applic^le across 
groups? The fust, internal validity, is a strength of qualitative methods, wmt the second, 
like reliability, pi^ difficulties for tte qualitative researdier who is oto embe^icd in a 
single context They charurterize qualitative research as contextual, theoretically eclectic, 
ami comimrative and note that the same direats to validity noted by Canmbell and Stanley 
(1963) exist for research (tf this type yet additional threats grow out of these diaracteristics 
of qualitative research. Histoiy and maturation, obs^v^ effects, selection ami regres^cHi, 
mortality of subjects, and ^Hirious coitelusions tiireaten internal validity m qualitative 
studies as they do in others. In the case of external validity, however, t^ ssmH size of 
most qualitative studies rutes out the use of statistkal metiiods for d^enniinng 
^endizability. Instead, external validity is a function of compmOHUty (the extent to 
which die research is described in a way ti»t is sufficioidy detailed diat it permits 
comparison to other studies) ai»i trarv^rabUUy (the extent to which the research is 
theoretically gmuiuied in csder to be understood by other researchers). Selection, history, 
setting, and constnx^ dfects also impede external validity. Howe and Eisenhart (1990) caU 
for a single standard for qualitative and quantitative resefuch. They suggest five standiuds: 

1. The data collection techniques must fit ami be suitable fw 
answering the research questions. 

2. Data collection and analysis techniques must be con^petently used 
in a technical sense. 

3. Studies must be jtsiged against a background o^* cristent 
knowledge. 

4. Confirming and disconflrming evidence must be 'ised :o assure 
an overall warrant <rf trustw(»thiness. 

5. Attention must be given to extomal value constraints u> the fcam 
of responses to *'So what?** qu^ticms and intonal ot ethkal value 
constraints should be attended to in osder to ensure the 8ccq)tability and 
legitimac) of research designs and the treatment ^ human subjects (pp. 6~ 
8). 

Perhaps the best known critoia fo^ ass^ing the validity or adequacy of qualitative 
research are those by Lincoln and Guba (1985) who established four major criteria for 
naturalistic inquiry: cndibUity w "troth value''(anivitics in the field that increase CTcdibility 
such as protonged engagement, persistent iH>savation and triangulation, pea* deteiefing, 
member dtecki, negative case analysis, ami refermtial adequacy), o^an^emlHUty cr 
applicability (titrough thick desrripiion), deperukMity or consistency (thnvu^ an audit trail 
of methods used to reduce data), and a»ifirtmbility or neutrality (Um)ugh an audit trail and 
objective description). Schatzman and Strauss (1973) offer three criteria for assessing the 
"credibility" or the "ring of trudi** for field research: propositions are grounded in data; 
vernation is made by the host organization or by an indqjmdent observer, and die 
phenomenon as described is r&:ogni2abk as "Uved experietuie*' (see Van Mancn, 1990). 

Vfl/k/tlry in action research, Cunningham (1983) en^hasazes that the tesirf validitv 
in action research is whedier <^ not the problem in the actimi context is scdved and locally 
valid" (p. 405). This ccmtextual validation is similar to the Chein, Code, and Harding 
(1948) idea that the job description of acticHi researchers requires that they not only tnake 
discoveries, but also see diat these discoveries are applied. In other words, the research is 
valid if it is applied in a particular context and "works" to solve the original presenting 

Winter (1987) discusses validity in action research finom a critical social sciaioc 
per^tecttve. To him, validity in action research is in the intcipr^e process rather titt i in 
particular interjwetations, a process tiiat is b(«h reflexive and dialectic. Accounts erf such 
research recognize tiiat "die epistemological adequacy of an account is described in terms of 
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the inteipcrsonal c<Miditions of its p(»sible prodiKtion'' (p. 126). Reflexive interpretation is 
the language ctf questions which asks whether any single intetpretation is valid when 
meanii^ are in pe(»le and socially derived. "It creates a t}^a:eiic space by nieans a 
geiwfal withdrawal nom interpretation to problematic. This is a ^mce diodirae within 
which discourse can proceed under the au^K^s of tl^eoretic grounds, which may be 
shared, and which thus may comt to be agreed as xilid theoretk grounds for the whole set 
of intensetations at issi^** (p. 12f >. Refintaice to the '^flexive pn^onatics of language 
itselT and an awareness that the action researcher is also beset by the veiy problematics 
he/^ hopes to describe is an essoitial element of a valid account (p. 134). A dialoE;tical 
approach asks us to onbrace (^>posing inteipretations and to [dace primaiy empha^ on 
meaning and umtotanding over reducing pnotomena to the tfaini^ oi the numerically 
ccnelated operational variaUe. Validity in action research, says Winto:, is not to be found 
in the (fegree of conespondence betwem reality and account, but rather in ^ theoretical 
use of the principles cfreflexivity and dialectics to discipline boA the telling of the account 
and the u*.lling ^ the pio^ss of umlentandtng the account In Winter, as in Argyris and 
Schon, it is the quality of the inquiry proc^ and the int^personal skill of the action 
resefucl^r which determine the validiw of d»ir accounts. Yet, to be consistent with tTitical 
social science, Lather (in Anderson, 19S9) suggests that the emancipatray interests of 
critical action research mu^ frame another aiteria for action research: caustic validity or 
the degree to which the respondents achieve further se^-understanding and setf- 
determination through participation in the research. 

For example, critiquing the ^;tion research project conducted by Lazes at Xerox, 
Aigyris and Schcm (1991) note that impropriate rig^ also involves a critical test of the 
ckums the research^. Perhaps the interventions ''work" because other conditions in the 
context (e.g. financial exigency) force than to and hem;e they are not likely to work again 
in otti^ ccmtexts. They call fm', at a mimmmn, an operational (Ascription of what the 
researcher actually did, and a critical reflection on the claims o€ attributions he or she makes 
about the achievements of the process. In this way, the research can be replicated and 
conm^g explanations (the negative case example) may be exanuned fin' the research 
results. They suggest that from the acticm researcho's p»:q>ective, "the challenge is to 
define and meet standards of apprc^mate rigoc without sacrificing relevance. Aiid, for this 
purpose, action research needs three things: a way of represen^ng research results that 
enhances their usability, a complementary way construing cauxility, and an appropriate 
methodology of causal irtference*" (p. 85). Commenting on validity and teacher inferences. 
House, Mathison, and McT^ait (1989) also call for an appropriate nKtlKxl of causal 
inference as a basis for validity in action research. Validity based on causal inferences that 
emerge from a theory of intentional causadon (cause and dffect relationships that 
practitioners can infier from their experiences), the kind that guide evoyday behavior, arc 
more iq)|Hopriate to action research than the Campbell et aL validity that is undergirded by 
the regularity theory of causation (in wMch research seeks to discover universal causal 
regularities). 

Action Science 

Aigyris has evolv^ over fcHty yrars of scholarship an apprc^h to action research 
entitled action science, a temi he credits to one of his former students, William Torbert 
(Aigyiis, 1980). Action science is a science of interpersonal action in which actors within 
a community of social practice reflect critically on their beliefs, values and actions and 
generate new knowledge and actions. Data collection in action science consists of the 
recollections of actions and speech by the actors, the reasoning and rules governing their 
actions, obs^ations of their ^ons, and action experiments. Data analysis in action 
sciem^ consists of discovering the meanings embedded in the action and representing the 
knowledge thus acquired in a way that is b^ disconfiimable and actionable. Data analysis 
tools include die use of the abstraa models which Argyris contends represent two 
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contrasting and genemlizable apprc^aches to action (Moctels I aiKi D), a ladder of inference^ 
in which the movenient fivm data to inference is made explicit and challengable, and 
maj^g^ which is a dquction of the causal modd governing action in individuals or 
cffganizad(»is. Aigyris (1980) contends that rigorous research {mxiuced by nonnal science 
is flawed in that it uses a control paradigm that leads to t^jendence on the part of subjects 
ami potoitially an automatic tacit pioc^ that may pn)dtx:e distorted perceptions wten 
individuals (cither subjects or rescardiCTS) are unaware of gaps between what they espouse 
and what they do. Ar^ris argues instead f(x validity in the form of data which is verifiable 
(dbectly observable) and interpretations which permit fuhire action to improve the problem 
situations and which are thus disconfirmable based on subsequent tests in the acti(»i 
context Moreover, like Winter, he asks that the causal inferences, both of subjects and 
researchers, be made explicit Finally, he a^ that we devel(^ a sense of validity about 
action knowledge in face-to-face situations that is both ethical (we do not unknowingly 
produce either deperuknce or distonion) and usable. 

Summary Validity Oiteria in action research 
To summarize, many criteria have been pn^sed to assess the trustwcHihiness of 
action research studies. Criteria tend to evolve from either a cou^ to action research 
(qualitative, nanuiaUstic research), or ftcm re»:iions against validity in the CarmjbeU et aL 
understanding which is characterized as "normal" science. Criteria onanatlng nom this 
review are those which are a r^ult of the actk>n-orientation and the fact that in this 
research, ths skill of the interventionist/ researcher is a key determinant in the quality of the 
results. Since the interventionist must produce the phenomena he or she wishes to study, 
thoe is an intent need for a hi^ level of inierp»sonal ccmipetence and for data cm the 
resmehet's practice. This must be otnained while »multaneously observing others. 
M(»eover, the pviicipants in action research are expected to take action as a result of this 
work. Hence, msights and fmdings are presented in a fonn that permits future action as 
well as future testing. The dilemma of all researchers reganLng how to enhance knowledge 
utilization is exacxibated by the iterative acticMi r^earch process whk;h cycles continuously 
between producing knowloige and taking Ktion on that knowledge. Only the critical 
action researchen and Ai:gyris and Schon sppeax to »;knowledge the de^ee to which this 
action taking is dependent not cmly on consnunitaticBi of the data but also on the ability of 
the participants to leam and apply new skills. 

Other validi^' criteria ewergj^ from the participatory nature of action research and 
action science studies, including the extent and nature or quality of partidpaticm ^Ekien and 
Levin, 1990). Many action research studies appear to abort at the stage of diagnosis of a 
problem or the implementation of a single solution strategy, irre^)ective of whe^er or not it 
resolves the presenting problem. Validity both process and of outcome must be 
established. Most of tbe criteria rBC(»ninended by the authors reviewed hoe continue to 
emphasize action o^ research which is the major reason n^thodologists have tended to 
suggest that action research is not really research at all but a kind of applied problem- 
solving i»x)cess. If action research is going to be accepted as science, validity criteria will 
have to demonstrate validity in both action and irsearch communities. Only Argyris 
appears to do this in that he adds to the preceding criteria the idea of generalizability of the 
knowledge produced in action research, specifically the biowledge of intervoition or of 
interpersonal action. Table one contrasts criteria for action research with those suggested 
by Lincoln and Cuba (1985). 



^This is his method of causal inference. Argyris wpicis throt^h Utis means a causal reo^ng i»ocess 
which leads to incompetent actkm. 

^yiappm$ is a form of data di^lay which normatively suggests altemaiive action suategies to those wh^ 
have led to itsulis that the individual or organization i^greear^ SeeFigurt2 
for an example. 
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-Insm Table 1 about here- 
Implicit Validity Criteria in Two Studies 

Another source inrcaniation about C(h»^ of validity is that implicit in the 
deagn ami conduct c^acticm research. Findings firom two studies will be used to iUusti?*« 
additional validity crit^ ami issues for action leseareh. A clasiuc study was that b^ Lewin 
(19S2) who described an action rese2ax:fa project to change the eating hatnts of individuals 
during tte war to inciease individuals' consumption of unpopular but nutritious foods surb 
as beef hearts, swee^rtads, and kkimys. He initially ccHlected data todq)ict the process 
of moving food to the family table. He determined ths! housewives woe the prmiaiy 
decision-makers reguding eating habits. Radke and Klisurich (in Lew?", 1952) found that 
coup dianisaon was thirty times ttm likely to increase consumptbn of the taiieted foods 
^%vs.32%). Subsequent stut&es to encourage milk consumption showed a similar 
relationship; and group decision also proved nxHt dfkacious than individual instruction in 
encouraging consumptioa of ^ange juice and cod liver oil 

The research reported by Lewin was validated in at least two explicit ways. First, it 
was assun^ that the change would have to be long term, hence persistence of the change 
was a criterion for evaluating the effectiveness of the action reseaidi project This led to 
repeated measurement of the change through follow-up contacts with the housewives in 
both groups, a researdi device which might also have been an uninientional initrvention of 
its own to encourage use. Second, the lesrarch was validated by measuring the degree of 
accomplishment of tlw original gc»l, consmnption of the specified foods. 

Lewin explained the results of this cdlection of studies in terms psychological 
ies»onses to infli^nce when an individual is resistant to an idea or to change. His theory 
of how forces in the life space resporal to the removal of constraints <x toners to 
movement ami to group influence was the theoretical fmmewoik for these studies. The 
group intervention and the lecture were specifically structured to at least potentially enable 
tl» renioval of constraints. Theory guicfed both intervention and analysis. In many ways, 
Lewin repacts studies that are like ntiost otter experimental studies with the only differences 
being the quality of on-line interactive engagement with the person whose behavicH- the 
resemchers seek to change and the presence of a larger social change objective. The 
metibods Lewin reports were not collaborative or participative nor was tlWie an explicit 
critical or emandpatcsy purpose. I incliKle this well known study in this analysis to 
suggest that in nmny ways, Lewinian action research is mcae r^earch-txiented than action- 
oriented, yet action in the fomi of a social change (changing eating habits to support the 
war effort) was the over-arching gc^. Lewin's work differs condderably from more 
contemporary emancipatoiy variants though research is stUl in the service of action. 

Action science is one fomi of action r^arch which inc€«p«ates an emancipatory 
intent and illustrates the concommitant problems of validity of this approach. In an action 
science study, this author examined the idea of empowerment for supervisors in a Fortune 
100 high technology corporation. Thirty supervisors from three different shifts and two 
separate divisions were divided into six case study groups. The smdy was an examination 
Ot critical incidents of problematic inter^ons in the workplace around the theme of 
empowerment. Supervisors wrote cases of problem incidents with both what they said or 
did and what they thought but did not say. Groups consisted of five or six first or second 
line supervisors plus an action science fadlitator/researcher. Group discussions then 
explored the nature of the thinking <x reasoning behind the actions taken, pos^ble 
unintended consequences of the actions taken, and feasible alternative options for action. 
These ^ups met over a period of four months which ena'aled them to experiment with 
alternative acti(»i strategies to begin to implement some of the ideas emerging about 
empowerment Dam in the form of cases (at least one from each participant), transcripts of 
initial i»c^lem identification interviews, and transcripts of group discussions were 
collected. The study concluded with a presentation of a number of maps depicting critical 



Watldns, K.,Validity in Action Research 



9 



themes in empowoment to the q)eratumal managers to whom the supovisors reported. 
Figure 1 is an example of a »cond level supmisor's case which iUustni»s die high 
Cf^sequences supervisors poceived in malong mistake. The map devek^ to ponray 
the theme oi making mistakes in a ujo defect culture is presented in Hgure 2. 

-Insen Figures 1 and 2 about here- 
Maroin| is a vital instnin»nt in action science. Maps allow action scientists to go 
beyond the (tetails of a particular case to represent pattems d behavior and belief systems, 
lliey are systematic dqnctions of govoning values, action ^rategi^ and consequences. 
No map is oon^lete until partidjmnts c^fier confirming and disconfmning evidence 
regarding the m^. In this case, each of the six groups confirrned ot disconfirmed each of 
the maps depicting the themes of making xmstakes, new rules or values, responsibility, 
centralization, and learning in the woikiNace. In this process, mines' changes and 
sonnetimes major additions were made to the maps. This manber checking is one fomi of 
valklity in the action science ]»t)cess, but here there is the added assumption that 
participants have internalized {Ironed thn»i^ tl» action sdenoe process) values and beliefs 
ag£unst which new data is tested. Validly is thus partially detomined by die skill of the 
facilitator in teaching new skills to participants. Tlie n^p is an heuristic, artistic 
embodiment of the themes identif»d tfannigh the ^roup analysis of cases. Devdopment of 
a map takes condderable skill and concepti^ clanty about the theory of inteiper»7nal acdon 
endxxlied in action science, particularly since it is inteiKled to be another intervention in the 
action science process. The relationslup to the original data, the cases present^ must be 
clear. The map depicts a causal pattern of belies and actions which are alts^le, hence the 
idea that research results are presented in an actionable or usable form. When presented in 
a normative format, alternative beliefs and actions are suggested in contrast to those that 
emerged from case discusskjns. Thoefore another source of validation is whether or not 
subsequent new actions alter the original pn^lem posed in tte direction hypothesized by 
the alternate m^. Figure 3 is an example of a nomiative map used in diis stikly. 

"Insert Figure 3 about here- 

Theory to Practice Gaps 
Does validity of process or diagnosis necessarily prodiwe validity of outame? In 
the comments of action researchers, one sees a number of dilemmas surrounding validity. 
On the one hand, diagnosis is easier to set up as a more "controUed" research process, so 
the researcher often experais more time and en^gy in this area leaving the actual 
implementation of the action reccmimendations up to participants who may or may not have 
acquired the skills to implem«it them. Construct validity in action r^earch terms is to 
some extent whether cs- ncH the outcomes developed actually solve the presenK»ci |m>blems. 
As in normal science research, this is the most difficult criterion. Further, action 
researdiers have difficulty communicating the extent to which the intavaitions they 
implcmait are skillful, other than in tcmB of their cwtcomes. This probably accounts for 
the plethora of "success story" action research reports. Yet, most action research projects 
arc long term cyclical processes of succ-'ssive approximations of tiie phenomena of interest. 
Where are the critical reports of the itiore problematic fits and starts of such efforts? The 
skill of diagnosis ami analysis in re;.' time is also difficult to communicate, but peiliaps 
greater detail about the early stages of an action research project would help novice action 
researchers begin to appreciate what is involved. In the end, these are highly personal 
skills. As Moustakas (1990) notes (paraphrasing Polanyi, 1969) "The synth^is of 
essences and meanings iitoent in any human experience is a reflection and outccsne of the 
researcher's pursuit of knowledge. What is presented as truth and what is removed as 
implausible or idiosyncratic ultimately can be accredited <»ily on the grounds of knowledge 
and judgment" (Moustakas, 1990, p. 33). Even the description of the project as a 
''success" seems to me to take a very complex mosaic of things that wwked, people who 
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learned and peq>le who did mm, and an overall i>rocess that is ongoing, and reduce this 
mosa^ to "the thinness the numerical conflation." 

Is is a qmsUon cf validity or ofgtm in skiU? As I reftect «i my own practice. I am 
struck by what seems like a minimizing <» the extraoidinaiy difficulty in devek^ng both 
one's own and participants' skill in enacting new behaviors suggested by the nodons of 
validity prcsaittd here. This teaming/teaching a^Jcct of m^oa research is intadcpendcnt 
andcontextuaL It is dear that oie reason action research studies are often diagnostic is that 
there is a gap between our ability to d»gm>se and invoit a solution and our ability to 
produce it (Ai^^s, Putnam, and Smith, 1985). In action research, this gap is Ukely to be 
in both the researcher and the researched. I recall the superviscar who came in to one group 
session with his equivalent to a "Eureka!" He reported that he had tried to use the i»w 
skills hi was learning to disdplii» an en^loyee. He stopped himself from lapsing into me 
of his finequent icn^ flare-ups and rememb«ed to listen to tte cnqjloyec and, while 
sharing his initial judgment about her behavujr, to iiMjuire into her reasoning about the 
incident He found that his new skills m^ her miwih rocHc receptive to his advice. In 
effect, from the pcr^iective of mrtion science, he had teamed a new way to control others, 
rather than a new pvadigm for empowering others. He had a tool, but not the underlying 
values behind the tod. Despite tus learning pxogress, th^ was still a significant gap 
between his inventions and his i^rtions. The extraordinary number of minute-to-minute 
judgment caUs between action and research imperatives may also lead to compromises of 
eitho* or both agendas. In fax^ to face real time con^ts, in which the ^>al is to teach 
people new bchaviws that are tiiemselves difficult cff hi^ threat, the interventionist must 
contiiiU?iiy respond to the evolving ability and willingness of the participant to change. 

Another skill gcp. Individuals seeking to solve problems in complex, real-tinw 
settings find that the problems change under their feet, often befc^ the more in-depth 
itcratfive search fw solutions suggested by action research has achieved meaningful results. 
For example, in the empowerment study, our initial contrM:ting included a requirement that 
all participants be volunteers. Midway in the isx)ces$, a few imlividuals alluded to the 
process by whfch tiiey "volunteoed." Managers had called them into their office or sent 
them a naemo asking them to "volunteer" fcH- this project In odier areas, a memo was sent 
to all supervisors asking that those interested contact persom^l to sign up for the program. 
It became clear that my competence in contracting with the organization was limited to an 
espoused value on voluniaryparticipation but I had left it up to the organization to 
determine how to enact it The obvious contradictions between empowCTnwni and 
marKlatory participation clearly had the potential to undermine the ^oals of the action 
research project From a rraearch f«rspectivc, reluctant participants may be more Ukely to 
imluce evidence of the critical incidents of a lack of cm|>owcrment, incluiting the way in 
which they reached the sesaon. From an action perspective, the evidence of systemic 
barriers to empowtament was made explicit and these individuals were less Ukely to be 
convinced that they could alter their assumptions and attain peatcr control over their 
environment. Furthermore, a pan of the group structuring included a mixing of shifts and 
divisicMis, heiu:e many groups irurlud^ ii^dividuals frcnn the areas that had mandated 
participation. Although this issue '.ed to a number of discussions about the directly 
olmvable differences in empowennent from area to area in the organization, feelings of 
injustice seemed to increase among those areas that had mandated participation as evidenced 
in comknents like, "Well you can try tiiat in area but in sm area- • •** I could not help 
but think Uiat had my skill in contracting and teaching the client been greater, I might have 
increased the likelihood tiiat tfie action research itself would not be structured in advance in 
a way that was disempowering to at least some of those who participated. 

Generalizability vs. Contextual validity. Argyris and Lewin contend that the 
findings from action research can add to the body erf "action" knowledge. Argyris 
develq>ed Moctel I (a control paradigm) and Model II (a learning ot reflective paradigm) as 
graeraiizatle, contrasting depictions of interpersonal action which can be tested across 
many contexts with different people. Cases in action science are analyzed to determine the 
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extent to which the ji^Ieroatic interacfioii rqxesents a control orientation and individuals 
rde i^y the interaction white atten^ting to eniu:t a teaming mentati(Hi in oixter to test (by 
group reftection) whether or not the en»:tixient of Model II behavion produces more 
effective results. However, the enactnnent Model H bdiaviors is counterintuitive and 
requires significant, transfonnative learning. Again, the teaching/learning and interpersonal 
ccHnpetence of the researcher must be at a very high tevel bdort (me can truly test this 
assertion. 

Maps in acti«i science are intended to dq)ict patterns of reasoning and behavior in a 
given context Yet, I have found that ih^ may generalize to otho* c(»itexts as wdl. One 
map I developed in another study described the internalized belies of trainers about their 
role and illustrated actions which follo\m! from than. The map was based on intmaews 
with fifty-seven human resource developer in three diffoent types of organi^ons and 
was confumed by a subgroup of these individuals as well as by other groups cf human 
resomce developers. At a research confoence, a researcher who h^ recently ccni^eted 
interviews with a number d literacy educators tmed that this map described what she had 
seen in her research as weU. This Qrpe of validati(Xi wpears to me K> be at least an 
intimation that these aie generalizabte pattens beyond a given local conmt though of 
course in this instance the maps were developed in an interview study. The maps 
developed at the high technology ccapoiation woe validated across ax different groups, 
which again sugge^ at least linuted generalizability to the larger OTganizational ctmtext 
Moreover, the fact that I and B. Blaocbum-TUmer were able to adajH D. Smith's (1986) 
map on managing transitions to reflect issues of enqxywemoent with mly rmnoar 
adjustments suggests that Smith's map may be somewhat genendizable to otiier problems 
of OTganizationaJ change. At any rate, in work with now over 200 individuals I have seen 
that the constructs of Model I and Model n have recuningly depicted agnificantly different 
panems of behavior which produce significantly different results rqieatably. Surely this b 
goieralizability? 

Implicatitms 

Quantitative and qualitative research have shared a standard of objectivity. Yet, 
participants in action research programs expect to be treated not as objects ot even subjects, 
but as co-rcscarchers engaged ui "empowering partidpation" and in "co-graioative 
dialogue" between "insiders and outsiders" (Elden and Levin, 1990). As Howe and 
Eisenhart (1990) tM)te, standards f<»r ediK^titMial inquiry must go beyond mutological 
arguments for or against insider or outsider pmpectives. Rather they must be "anchored 
wholly within the process of inquiry" (p. 3) and address the bro^^T question "Are 
warranted conclusions obtained about some important educational questions?" (p. 6). In 
action research, truth "anchored wholly within the process of inquiiy" is in the process of 
inquiry itself. Was it reflodve and dialectical? Was it ethical and collabc^tive? Did 
participants leam new research skills, attain greater self-understanding, or achieve greater 
self-dctennination? Did it solve significant practice problems or did it contribute to our 
knowledge about what will not solve those probtems? Were problems solved in a manner 
that enhanced the overall learning capacity of the individuals or system? These are the 
types of questions which must guide action re^arch. 

Nevertheless, reviewers who equate intervention with practice, subjectivity wth 
bias or philosophizing, ch- heuristic forms of data display with "going beyond the data," 
continue to play a powerful rote in setting the standards for research and in shying the way 
such research is described in research journals. In 1947, Lewin wrote that "The research 
needed for social practice can best be characterized as research for social management or 
social engineering. It is a type of action-research, a comparative search on the conditicms 
and effects of various forms of social action, and research leading to social action. . > .This 
by no means implies that the research needed is in any respect less scientific or 'lower' than 
what would be required for pure science in Uie field of social events. I am inclined to hold 
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the q^josite to be tn»" (pp. 150- ISl). This papo" has reviewed the literature cS i^on 
reseaidi and actkm scieiK« to sugge^ comprang imd difto 

types of research. Lewin may indeed be correct in suggKting that apisopriate rigor in this 
reseaxy*h is more demanding. Finally, I have idoitiHed a numbor of gaps between the 
theoiy and the {vactice of validating action research projects which pcrint to a need to clarify 
and en^iasize the ^ificant n^e oit teaching/ieaming and interpersonal competeiK:e in 
producing valid, potentially even generalizabic lesults in action research. 
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Tabic 1. A Conroarisw' of Gufca and Unooln's Criteria Assessing /alidity in 
Naturalistic In^mry with Critnia for Validity in Action Research 



lOuba and Lincoln 



Triistworthinessi/Credibiiity 

(How can we establish confidence 
in the truth of the findings for 
b<^ the context and fox the 
respondents?) 



Tkan^erability/Applk^bility 
(How do we detomine the extent 
to which Uie finding^ 
have applicability with otho* 
contexts or subjects?) 

Dependalnlity/Consistency 
(How cun we determine whether 
the findngs would be repeated 
with the same or similar 
subjects or contexts?) 



Gxifiimability/Neutrality 
(To what extent are die findings 
determined by the subjects & 
the comiitions of the inquiiy 
and iK>t by the biases of the 
researcher?) 



Actim R^rarcn 



Trustworthiness/ Usability 

(How con we both establish 
confidoice in the ve^arch findings 
while develq)ing both mion and research 
stalls among parddpants and ensuring that 
the soludon wc^ in context?) 

Relevancy/Applicalnlity in G>ntext 
(How (b we (tetermine 
the relevance of findings 
to tlw needs of the problem 
context?) 

Dependability/CompeteiKv 

(To what extent arc we able 

to deteraune the adequacy of the process 

and are problems solved in a 

manner that permits ongoing 

teaniing of tl^ individual or 

system?) 

Nonnative/Reeducative, 
(How consistent are procedures and 
outcomes with the nonnative theory 
guifiing the research? In other words, 
do i^irticipants leam, are situations 
transfomied as predicted by the theory?) 



ERIC 
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Figure 1: Making Mistakes: A Second Level Supervises- in A Case of the Wrcmg Foim 

Operator fitom 4th shift was issued a notification of a pdicy violation fonn for being late 
man lunch. This should have been countrd as an occuirence and her pay docked instead 
of an abuse of personal privilege. She denies being late a! all and believes she doesn't 
deserve u> be written vp. 



Why did we u^ the wrong form! 
This is going to be bad. 



Supv: I checked into the situation and 
(me) we did use the incorrect fona 
You don't get a documented 
warning for this because it was 
a tardy from lunch, but it is an 
occurrence. 



Open Well, it's not corrccL Idwi't 

cheat (the csganizaticH)) and I have 
no intentif»i doing so. lean 
get you a witness to verify I 
wasn't late. 



Sure, who would admit she was 
late knowing they have to work 
in the same area? 



Now she's going to think, 
"they always sdck together" 



Supv: No, I don't need a witness. I 
(me) wouldn't put you (x any indi- 
vidual that works with you in 
thai positicsi. Are you saying 
that your supenrisor is lying 
about the total time you were 
out the area? 

Open I am not calling my supervisor 
a liar, but I wasn't late and I 
can get you a witness. 

Supv: Well, I can't have a supervisor 
(me) lie abut something like this. 

There is no reason for this to 

be made up and ... 



C>per I understand you have to take 
his side, but I want you to 
know I'm taking this higher, 
not with anyone hcrt. I won't 
stop here. 

"HiREATf Supv: That is your prerogative and if 

you need any other information 
from me, let me know. Thank 
you for your time. 



1 
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HguitZ Making Mistakes: A Causal Map 



Beliefs 



Actions 



Consequences 



When I believe 
my subordinate 
made a mistake 
and that manage- 
ment will lose face 



Management should 
have ^ the details 
or ti»y mate wrong 
decisions and lose 
face. 



•Unilaterally 
{Hc^ect manage- 
ment by backing 
up the suboidinate 
and escalating the issue 

•Stand on Procedures, 
acting as ttough 
this were not a 
gray area which 
requires judgment 
calls 



•Act as though 
management has 
all ('€ details and 
do not inquire 
even when I 
know they aie 
wrong. 

•Blame upper 
managenient 
when operators 
point out errors 
in policy. 



Upper manage- 
ment is mcHe 
likely to over- 
turn the (kci»on; 



Hie subordinate 
will not leam 
how to make 
nxHe effiective 
judgment calls 
and I and 
management 
will lose face. 

Managers lose 
face. 

Empl(^ees are 
not mpowered 
to panicipate in 
a healthy way in 
(tecisions that 
directly affect 
them even when 
they have more 
information. 

Managers are 
not aware that 
en^loye^ have 
anything u> 
contribute. 
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Figure 3. MANAGING TRANSITIONS: TOWARD EMPOWERMENT 1 
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